Stimulus-outcome associations are required for the expression of specific Pavlovian-instrumental transfer.

A series of experiments employed a specific Pavlovian-instrumental transfer (PIT) task in rats to determine the capacity of various treatments to undermine two outcome-specific stimulus-outcome (S-O) associations. Experiment 1 tested a random treatment, which involved uncorrelated presentations of the two stimuli and their predicted outcomes. This treatment disrupted the capacity of the outcome-specific S-O associations to drive specific PIT. Experiment 2 used a negative-contingency treatment during which the predicted outcomes were exclusively delivered in the absence of their associated stimulus. This treatment spared specific PIT, suggesting that it left the outcome-specific S-O associations relatively intact. The same outcome was obtained in Experiment 3, which implemented a zero-contingency treatment consisting of delivering the predicted outcomes in the presence and absence of their associated stimulus. Experiment 4 tested a mixed treatment, which distributed the predicted outcomes at an equal rate during each stimulus. This treatment disrupted the capacity of the outcome-specific S-O associations to drive specific PIT. We suggest that the mixed treatment disrupted specific PIT by generating new and competing outcome-specific S-O associations. By contrast, we propose that the random treatment disrupted specific PIT by undermining the original outcome-specific S-O associations, indicating that these associations must be retrieved to express specific PIT. We discuss how these findings inform our theoretical understanding of the mechanisms underlying this phenomenon. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

High fat diet allows food-predictive stimuli to energize action performance in the absence of hunger, without distorting insulin signaling on accumbal cholinergic interneurons.

Obesity can disrupt how food-predictive stimuli control action performance and selection. These two forms of control recruit cholinergic interneurons (CIN) located in the nucleus accumbens core (NAcC) and shell (NAcS), respectively. Given that obesity is associated with insulin resistance in this region, we examined whether interfering with CIN insulin signaling disrupts how food-predictive stimuli control actions. To interfere with insulin signaling we used a high-fat diet (HFD) or genetic excision of the insulin receptor (InsR) from cholinergic cells. HFD left intact the capacity of food-predictive stimuli to energize performance of an action earning food when mice were tested hungry. However, it allowed this energizing effect to persist when the mice were tested sated. This persistence was linked to NAcC CIN activity but was not associated with distorted CIN insulin signaling. Accordingly, InsR excision had no effect on how food-predicting stimuli control action performance. Next, we found that neither HFD nor InsR excision altered the capacity of food-predictive stimuli to guide action selection. Yet, this capacity was associated with changes in NAcS CIN activity. These results indicate that insulin signaling on accumbal CINs does not modulate how food-predictive stimuli control action performance and selection. However, they show that HFD allows food-predictive stimuli to energize performance of an action earning food in the absence of hunger.

Influence of key histological characteristics on 18F-fluorodeoxyglucose /18F-choline positron emission tomography positivity in hepatocellular carcinoma: A machine learning study.

Purpose: To determine the characteristics influence of key histological on 18F-fluorodeoxyglucose (18F-FDG) and 18F-choline positron emission tomography (PET) positivity in hepatocellular carcinoma (HCC). Materials and methods: The 18F-FDG/18F-choline PET imaging findings of 103 histologically proven HCCs (from 62 patients, of which 47 underwent hepatectomy and 15 received liver transplantation) were retrospectively examined to assess the following key histological parameters: Grade, capsule, microvascular invasion (mVI), macrovascular invasion (MVI), and necrosis. Using a ratio of 70/30 for training and testing sets, respectively, a penalized classification model (Elastic Net) was trained using 100 repeated cross-validation procedures (10-fold cross-validation for hyperparameter optimization). The contribution of each histological parameter to the PET positivity was determined using the Shapley Additive Explanations method. Receiver operating characteristic curves with and without dimensionality reduction were finally estimated and compared. Results: Among the five key histological characteristics of HCC (Grade, capsule, mVI, MVI, and necrosis), mVI and tumor Grade (I-III) showed the highest relevance and robustness in explaining HCC uptake of 18F-FDG and 18F-choline. MVI and necrosis status both showed high instability in outcome predictions. Tumor capsule had a minimal influence on the model predictions. On retaining only mVI and Grades I-III for the final analysis, the area under the receiver operating characteristic (ROC) curve values were maintained (0.68 vs. 0.63, 0.65 vs. 0.64, and 0.65 vs. 0.64 for 18F-FDG, 18F-choline, and their combination, respectively). Conclusion: 18F-FDG/18F-choline PET positivity appears driven by both the Grade and mVI components in HCC. Consideration of the tumor microenvironment will likely be necessary to improve our understanding of multitracer PET positivity.

Second-order fear conditioning involves formation of competing stimulus-danger and stimulus-safety associations.

How do animals process experiences that provide contradictory information? The present study addressed this question using second-order fear conditioning in rats. In second-order conditioning, rats are conditioned to fear a stimulus, S1, through its pairings with foot-shock (stage 1); and some days later, a second stimulus, S2, through its pairings with the already-conditioned S1 (stage 2). However, as foot-shock is never presented during conditioning to S2, we hypothesized that S2 simultaneously encodes 2 contradictory associations: one that drives fear to S2 (S2-danger) and another that reflects the absence of the expected unconditioned stimulus and partially masks that fear (e.g. S2-safety). We tested this hypothesis by manipulating the substrates of danger and safety learning in the brain (using a chemogenetic approach) and assessing the consequences for second-order fear to S2. Critically, silencing activity in the basolateral amygdala (important for danger learning) reduced fear to S2, whereas silencing activity in the infralimbic cortex (important for safety learning) enhanced fear to S2. These bidirectional changes are consistent with our hypothesis that second-order fear conditioning involves the formation of competing S2-danger and S2-safety associations. More generally, they show that a single set of experiences can produce contradictory associations and that the brain resolves the contradiction by encoding these associations in distinct brain regions.

Basal forebrain cholinergic signaling in the basolateral amygdala promotes strength and durability of fear memories.

The basolateral amygdala (BLA) complex receives dense cholinergic projections from the nucleus basalis of Meynert (NBM) and the horizontal limb of the diagonal band of Broca (HDB). The present experiments examined whether these projections regulate the formation, extinction, and renewal of fear memories. This was achieved by employing a Pavlovian fear conditioning protocol and optogenetics in transgenic rats. Silencing NBM projections during fear conditioning weakened the fear memory produced by that conditioning and abolished its renewal after extinction. By contrast, silencing HDB projections during fear conditioning had no effect. Silencing NBM or HDB projections during extinction enhanced the loss of fear produced by extinction, but only HDB silencing prevented renewal. Next, we found that systemic blockade of nicotinic acetylcholine receptors during fear conditioning mimicked the effects produced by silencing NBM projections during fear conditioning. However, this blockade had no effect when given during extinction. These findings indicate that basal forebrain cholinergic signaling in the BLA plays a critical role in fear regulation by promoting strength and durability of fear memories. We concluded that cholinergic compounds may improve treatments for post-traumatic stress disorder by durably stripping fear memories from their fear-eliciting capacity.

Response-independent outcome presentations weaken the instrumental response-outcome association.

The present article explored the fate of previously formed response-outcome associations when the relation between R and O was disrupted by arranging for O to occur independently of R. In each of three experiments response independent outcome delivery selectively reduced the R earning that O. Nevertheless, in Experiments 1 and 2, the R continued to show sensitivity to outcome devaluation, suggesting that the strength of the R-O association was undiminished by this treatment. These experiments used a two-lever, two-outcome design introducing the possibility that devaluation reflected the influence of specific Pavlovian lever-outcome associations. In an attempt to nullify the influence of these incidental Pavlovian cues Experiment 3 used a single bidirectional vertical lever that rats could press left or right for different outcomes. Again, response-independent outcome presentations selectively depressed the performance of the R that delivered the response-independent O. However, in this situation, the response independent O also reduced the sensitivity of R to outcome devaluation; whereas the nondegraded R was sensitive to devaluation, the degraded R was not. We conclude that selective degradation of the instrumental contingency can weaken a specific R-O association while leaving other R-O associations intact. Furthermore, the use of a bidirectional vertical lever in Experiment 3 revealed that unidirectional and spatially separated instrumental manipulanda, such as levers or chains, may produce Pavlovian cues capable of forming incidental associations with the instrumental outcome that can obscure the relative influence of R-O associations after various manipulations. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Sensory-Specific Satiety Dissociates General and Specific Pavlovian-Instrumental Transfer.

Pavlovian conditioning enables predictive stimuli to control action performance and action selection. The present experiments used sensory-specific satiety to examine the role of outcome value in these two forms of control. Experiment 1 employed a general Pavlovian-instrumental transfer design to show that a stimulus predicting a food outcome energizes the performance of an instrumental action earning another food outcome. This energizing effect was removed when the stimulus-predicted outcome or a novel outcome was devalued by sensory-specific satiety. Experiments 2 and 3 employed a specific Pavlovian-instrumental transfer design to demonstrate that a stimulus predicting a particular food outcome promotes the selection of an instrumental action earning the same, but not a different, food outcome. Remarkably, this effect was maintained when all or just one of the stimulus-predicted outcomes were devalued by sensory-specific satiety. These results indicate that satiety alone removes the expression of general PIT. By contrast, satiety or outcome-specific devaluation does not regulate the expression of specific PIT, which is insensitive to changes in outcome value. This dissociation is consistent with the view that general and specific PIT are two separate phenomena driven by distinct psychological mechanisms.

The neural substrates of higher-order conditioning: A review.

Sensory preconditioned and second-order conditioned responding are each well-documented. The former occurs in subjects (typically rats) exposed to pairings of two relatively neutral stimuli, S2 and S1, and then to pairings of S1 and a motivationally significant event [an unconditioned stimulus (US)]; the latter occurs when the order of these experiences is reversed with rats being exposed to S1-US pairings and then to S2-S1 pairings. In both cases, rats respond when tested with S2 in a manner appropriate to the affective nature of the US, e.g., approach when the US is appetitive and withdrawal when it is aversive. This paper reviews the neural substrates of sensory preconditioning and second-order conditioning. It identifies commonalities and differences in the substrates of these so-called higher-order conditioning protocols and discusses these commonalities/differences in relation to what is learned. In so doing, the review highlights ways in which these types of conditioning enhance our understanding of how the brain encodes and retrieves different types of information to generate appropriate behavior.

Affective Valence Regulates Associative Competition in Pavlovian Conditioning.

Evidence suggests that, in Pavlovian conditioning, associations form between conditioned stimuli and multiple components of the unconditioned stimulus (US). It is common, for example, to regard USs as composed of sensory and affective components, the latter being either appetitive (e.g., food or water) or aversive (e.g., shock or illness) and, therefore, to suppose different USs of the same affective class activate a common affective system. Furthermore, evidence is growing for the suggestion that, in competitive learning situations, competition between predictive stimuli is primarily for association with the affective system activated by the US. Thus, a conditioned stimulus (CS) previously paired with one US will block conditioning to another CS when both are presented together and paired with a different US of the same affective class, a phenomenon called transreinforcer blocking. Importantly, similar effects have been reported when steps are taken to turn the pretrained CS into a conditioned inhibitor, which activates the opposing affective state to the excitor that it inhibits. Thus, an appetitive inhibitor can block conditioning to a second CS when they are presented together and paired with foot shock. Here we show that the same is true of an aversive inhibitor. In two experiments conducted in rats, we found evidence that an aversive inhibitor blocked conditioning to a second CS when presented in a compound and paired with food. Such findings demonstrate that affective processes and their opponency organize appetitive-aversive interactions and establish the valences on which they are based, consistent with incentive theories of Pavlovian conditioning.

Acquisition and extinction of second-order context conditioned fear: Role of the amygdala.

Second-order fear conditioning has been demonstrated in protocols using discrete and simple stimuli, and much is now known about its behavioral and neural characteristics. In contrast, the mechanisms of second-order conditioning to more complex stimuli, such as contexts, are unknown. To address this gap in our knowledge, we conducted a series of experiments to investigate the neural and behavioral characteristics of second-order context fear conditioning in rats. We found that rats acquire fear to a context in which a first-order conditioned stimulus is presented (Experiment 1); neuronal activity in the basolateral amygdala (BLA) is required for the acquisition (Experiment 2) and extinction (Experiment 3) of second-order context fear; second-order context fear can be reduced by extinction of its first-order conditioned stimulus associate (Experiment 4); and that second-order fear reduced in this way is restored when fear of the first-order conditioned stimulus spontaneously recovers or is reconditioned (Experiment 5). Thus, second-order context fear requires neuronal activity in the BLA, and once established, tracks the level of fear to its first-order conditioned stimulus-associate. These results are discussed with respect to the substrates of second-order fear conditioning in other protocols, and the role of the amygdala in different forms of conditioning.

General Pavlovian-instrumental transfer tests reveal selective inhibition of the response type - whether Pavlovian or instrumental - performed during extinction.

The present experiments examined whether extinction of a stimulus predicting food affects the ability of that stimulus to energize instrumental performance to obtain food. We first used a general Pavlovian instrumental transfer (PIT) paradigm in which rats were first given Pavlovian conditioning with a stimulus predicting one type of food outcome and were then trained to lever press for a different food outcome. We found that the Pavlovian stimulus enhanced performance of the lever press response and that this enhancement was preserved after extinction of that stimulus (Experiment 1) even when the context was manipulated to favor the expression of extinction (Experiment 2). Next, we assessed whether extinction influenced the excitatory effect of a stimulus when it was trained as a discriminative stimulus. Extinction of this stimulus alone had no effect on its ability to control instrumental performance; however, when extinguished with its associated lever press response, discriminative control was lost (Experiments 3 and 4). Finally, after instrumental and Pavlovian training, we extinguished a Pavlovian stimulus predicting one food outcome with a lever press response that delivered a different outcome. In a general PIT test, we found this extinction abolished the ability of the Pavlovian stimulus to elevate responding on a lever trained with a different outcome, revealing for the first time that extinction can abolish the general PIT effect. We conclude that extinction can produce an inhibitory association between the stimulus and the general response type, whether Pavlovian or instrumental, performed during the extinction training.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer.

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Basolateral Amygdala Drives a GPCR-Mediated Striatal Memory Necessary for Predictive Learning to Influence Choice.

Predictive learning exerts a powerful influence over choice between instrumental actions. Nevertheless, how this learning is encoded in a sufficiently stable manner to influence choices that can occur much later in time is unclear. Here, we report that the basolateral amygdala (BLA) encodes predictive learning and establishes the memory necessary for future choices by driving the accumulation of delta-opioid receptors (DOPRs) on the somatic membrane of cholinergic interneurons in the nucleus accumbens shell (NAc-S). We found that the BLA controls DOPR accumulation via its influence on substance P release in the NAc-S, and that although DOPR accumulation is not necessary for predictive learning per se, it is necessary for the influence of this learning on later choice between actions. This study uncovers, therefore, a novel GPCR-based form of memory that is established by predictive learning and is necessary for such learning to guide the selection and execution of specific actions.

The role of the basolateral amygdala and infralimbic cortex in (re)learning extinction.

The basolateral amygdala complex (BLA) and infralimbic region of the prefrontal cortex (IL) play distinct roles in the extinction of Pavlovian conditioned fear in laboratory rodents. In the past decade, research in our laboratory has examined the roles of these brain regions in the re-extinction of conditioned fear: i.e., extinction of fear that is restored through re-conditioning of the conditioned stimulus (CS) or changes in the physical and temporal context of extinction training (i.e., extinction of renewed or spontaneously recovered fear). This paper reviews this research. It has revealed two major findings. First, in contrast to the acquisition of fear extinction, which usually requires neuronal activity in the BLA but not IL, the acquisition of fear re-extinction requires neuronal activity in the IL but can occur independently of neuronal activity in the BLA. Second, the role of the IL in fear extinction is determined by the training history of the CS: i.e., if the CS was novel prior to its fear conditioning (i.e., it had not been trained), the acquisition of fear extinction does not require the IL; if, however, the prior training of the CS included a series of CS-alone exposures (e.g., if the CS had been pre-exposed), the acquisition of fear extinction was facilitated by pharmacological stimulation of the IL. Together, these results were taken to imply that a memory of CS-alone exposures is stored in the IL, survives fear conditioning of the CS, and can be retrieved and strengthened during extinction or re-extinction of that CS (regardless of whether the extinction is first- or second-learned). Hence, under these circumstances, the initial extinction of fear to the CS can be facilitated by pharmacological stimulation of the IL, and re-extinction of fear to the CS can occur in the absence of a functioning BLA.

The conditions that regulate formation of a false fear memory in rats.

People and animals sometimes associate events that never occurred together. These false memories can have disastrous consequences, yet little is known about the conditions under which they form. In four experiments, we investigated how rats learn to fear a context in which they have never experienced danger (i.e., how they form a false context fear memory). In each experiment, rats were pre-exposed to a context on day 1, shocked in a similar-but-different context on day 2, and tested in the pre-exposed or explicitly-conditioned context on day 3. The results revealed that: (1) the true memory of the explicitly-conditioned context and false memory of the pre-exposed context develop simultaneously and independently; and (2) the conditions of pre-exposure on day 1 and time of shock exposure on day 2 interact to determine the strength of the false memory. These findings are anticipated by a recent computational model, the Bayesian Context Fear Algorithm/Automaton (BACON; Krasne, Cushman, & Fanselow, 2015). They are discussed in relation to this model and more general theories of context learning.

Role Played by the Passage of Time in Reversal Learning.

Reversal learning is thought to involve an extinction-like process that inhibits the expression of the initial learning. However, behavioral evidence for this inhibition remains difficult to interpret as various procedures have been employed to study reversal learning. Here, we used a discrimination task in rats to examine whether the inhibition produced by reversal learning is as sensitive to the passage of time as the inhibition produced by extinction. Experiment 1 showed that when tested immediately after reversal training, rats were able to use the reversed contingencies to solve the discrimination task in an outcome-specific manner. This ability to use outcome-specific information was lost when a delay was inserted between reversal training and test. However, interpretation of these data was made difficult by a potential floor effect. This concern was addressed in Experiment 2 in which it was confirmed that the passage of time impaired the ability of the rats to use the reversed contingencies in an outcome-specific manner to solve the task. Further, it revealed that the delay between initial learning and test was not responsible for this impairment. Additional work demonstrated that solving the discrimination task was unaffected by Pavlovian extinction but that the discriminative stimuli were able to block conditioning to a novel stimulus, suggesting that Pavlovian processes were likely to contribute to solving the discrimination. We therefore concluded that the expression of reversal and extinction learning do share the same sensitivity to the effect of time. However, this sensitivity was most obvious when we assessed outcome-specific information following reversal learning. This suggests that the processes involved in reversal learning are somehow distinct from those underlying extinction learning, as the latter has usually been found to leave outcome-specific information relatively intact. Thus, the present study reveals that a better understanding of the mechanisms supporting reversal training requires assessing the impact that this training exerts on the content of learning rather than performance per se.

The infralimbic cortex encodes inhibition irrespective of motivational significance.

Evidence indicates that the infralimbic cortex (IL) encodes and retrieves the inhibitory memory produced by fear extinction. Recently, we have shown that the IL is also involved in the inhibitory memory generated by stimulus pre-exposure that causes latent inhibition. These results are surprising because a stimulus undergoing fear extinction carries aversive motivational value, whereas a pre-exposed stimulus is neutral. The present experiments tested the hypothesis that the IL encodes inhibition irrespective of the motivational information about the stimulus. Using rats, we first confirmed that IL activity during stimulus pre-exposure is required for latent inhibition. Then, we found that pharmacological stimulation of the IL facilitated aversive extinction to a stimulus that had been trained and extinguished as an appetitive stimulus. This facilitation was stimulus specific and required appetitive extinction. The same facilitation was found when appetitive extinction was replaced with random presentations of the stimulus and an appetitive outcome. Together, these findings indicate that non-reinforced stimulus presentations establish an inhibitory memory that is reactivated and strengthened in the IL during subsequent aversive extinction. This is consistent with the view that the IL encodes inhibition irrespective of motivational value, suggesting that this brain region plays a general role in inhibitory learning.

Motivational state controls the prediction error in Pavlovian appetitive-aversive interactions.

Contemporary theories of learning emphasize the role of a prediction error signal in driving learning, but the nature of this signal remains hotly debated. Here, we used Pavlovian conditioning in rats to investigate whether primary motivational and emotional states interact to control prediction error. We initially generated cues that positively or negatively predicted an appetitive food outcome. We then assessed how these cues modulated aversive conditioning when a novel cue was paired with a foot shock. We found that a positive predictor of food enhances, whereas a negative predictor of that same food impairs, aversive conditioning. Critically, we also showed that the enhancement produced by the positive predictor is removed by reducing the value of its associated food. In contrast, the impairment triggered by the negative predictor remains insensitive to devaluation of its associated food. These findings provide compelling evidence that the motivational value attributed to a predicted food outcome can directly control appetitive-aversive interactions and, therefore, that motivational processes can modulate emotional processes to generate the final error term on which subsequent learning is based.

The Lateral Habenula and Its Input to the Rostromedial Tegmental Nucleus Mediates Outcome-Specific Conditioned Inhibition.

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning.SIGNIFICANCE STATEMENT Stimuli that positively or negatively predict rewarding outcomes influence choice between actions that deliver those outcomes. Previous studies have found that a positive predictor of a specific outcome biases choice toward actions delivering that outcome. In contrast, a negative predictor of an outcome biases choice away from actions earning that outcome and toward other actions. Here we reveal that the lateral habenula is critical for negative predictors, but not positive predictors, to affect choice. Furthermore, these effects were found to require activation of lateral habenula inputs to the rostromedial tegmental nucleus. These results are consistent with the view that the lateral habenula establishes inhibitory relationships between stimuli and food outcomes and computes a negative prediction error in Pavlovian conditioning.